Smart Interactive Wall


Summary

A Smart wall is an avatar based open domain conversational chatbot built to engage the visitors of NUS. When a user comes in proximity with the smart wall, the smart wall gets activated. It captures the general information of the users such as age, gender and also identifies the emotional state of the user. It then begins to have conversation with the user.

Project Links

Architecture

Smart interactive wall identifies the user when he comes in proximity wall using kinect based .Net application. The smart wall operates in two modes - Information mode and generative mode. The green flow shown below shows the generative mode and orange flow represents the informative mode. In generative mode a deep learning model is trained to respond to user's voice input. The model is build using this tutorial.In informative mode a set of 10 basic categories were identified and trained on DialogFLow to respond to user's queries. To obtain general information the content is scraped from the search engines and natural language processing techniques are applied to answer users. The response are shown visually using an avatar which renders itself in the Unity framework.

Two Modes of Smart Wall

Two modes of Smart Wall

Flow of Information In Smart Wall

Flow of information

Demo

← Previous project
Next project →